feat: add Spark-compatible atan2 function - #23962
Conversation
|
cc @Jefffrey @comphead — a follow-up to my recently merged It follows the existing Covered by sqllogictest cases: all four quadrants, both axes, the negative x-axis (range to π), NULL propagation, NaN in every position (incl. Would appreciate a review when you have a moment. Thanks! |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #23962 +/- ##
==========================================
- Coverage 80.70% 80.70% -0.01%
==========================================
Files 1095 1096 +1
Lines 372554 372586 +32
Branches 372554 372586 +32
==========================================
+ Hits 300661 300683 +22
- Misses 53924 53929 +5
- Partials 17969 17974 +5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
how is this different from the datafusion functions ver? |
Good question. The element-wise math is the same underlying
So I think this is the same reason Happy to adjust if you'd prefer a different approach here or think otherwise. |
Which issue does this PR close?
datafusion-sparkSpark Compatible Functions #15914Rationale for this change
Spark provides
atan2(exprY, exprX), which returns the angle in radians between the positive x-axis and the point given by the coordinates (exprX,exprY). It was not yet implemented indatafusion-spark— only an auto-generated test stub existed atspark/math/atan2.sltwith its query commented out.What changes are included in this PR?
SparkAtan2(implementingScalarUDFImpl) indatafusion/spark/src/function/math/atan2.rs, computingy.atan2(x)element-wise via the Arrowbinarykernel.datafusion/spark/src/function/math/mod.rs.atan2.sltsqllogictest.The signature is
exact(Float64, Float64) -> Float64, following thedatafusion-sparkconvention of only accepting Spark-supported types. NULL in either argument propagates to NULL.Note: DataFusion core already has an
atan2, but this is implemented self-contained rather than delegating — matching the crate convention, where 14 of the 15 existingmathfunctions reimplement rather than wrap core (includingabs,ceil,floor,round,rint, all of which also exist in core; onlypowdelegates, as a special case). This keeps the Spark function library complete and consistent.Are these changes tested?
Yes —
datafusion/sqllogictest/test_files/spark/math/atan2.sltcovers:atan),atan2(NaN, Infinity) = NaN(for atan2, NaN propagates even over Infinity — unlikehypot, where Infinity dominates),atan2(-0, -1) = -π),Are there any user-facing changes?
Yes — adds the Spark-compatible
atan2scalar function todatafusion-spark. No breaking changes to public APIs.